System and method for location determination using movement of an optical label fixed to a bone using a spatial mapping camera

ABSTRACT

A system for determining a location for a surgical procedure, having a 3D spatial mapping camera, the 3D spatial mapping camera configured to map a bone. The system also includes a marker attached to a distal end section of the bone, such that the 3D spatial mapping camera is configured to capture a plurality of images of the marker as the bone is rotated in a non-linear path. The images also include data identifying a location of the marker. The system also includes a computer system that receives the data from the images captured by the 3D spatial mapping camera and determines a location of a mechanical axis of the bone, and a mixed reality display, where the computer system is configured to send the location of the mechanical axis to the mixed reality display and the mixed reality display is configured to provide a virtual display of the mechanical axis of the bone.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.17/562,917, filed on Dec. 27, 2021, which is a continuation of U.S.patent application Ser. No. 17/402,360, filed on Aug. 13, 2021, which isa continuation of U.S. patent application Ser. No. 17/221,760, filed onApr. 2, 2021, which is hereby incorporated by reference herein in itsentirety, including but not limited to those portions that specificallyappear hereinafter, the incorporation by reference being made with thefollowing exception: in the event that any portion of theabove-referenced application is inconsistent with this application, thisapplication superseded said above-referenced application.

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

Not applicable.

BACKGROUND 1. The Field of the Present Disclosure

The present disclosure relates generally to surgical systems and methodsof facilitating the efficiency and accuracy of implanting surgicalprostheses using fixed markers attached to an end portion of a bone,mixed reality and 3D spatial mapping devices.

2. Description of Related Art

In conventional knee replacement procedures, a femur is cut on a distalend which can then receive a knee replacement. Conventionally, ananatomical axis of the femur is identified by inserting a rod into thefemoral canal of the femur. The rod is inserted into the entire lengthof the femur and acts as an identifier for the anatomical axis of thefemur. A mechanical axis of the femur can be estimated as shifted atabout 5 degrees, or 5 degrees offset, from the anatomical axis of therod. A cut line, or cut plane, is then identified as perpendicular tothe mechanical axis. The mechanical axis is conventionally difficult toidentify in the operating room because of the engagement of the proximalend of the femur in the hip socket. However, this conventionalprocedure, inserting a rod into the entire length of the femur, is veryinvasive, which can cause a variety of complications and potentialinjury to the patient.

Once the mechanical axis is identified, the proper positioning of a jigor knee implant can be made. A femoral implant and tibial implant aretypically designed to be surgically implanted into the distal end of thefemur and the proximal end of the tibia, respectively. The femoralimplant is further designed to cooperate with the tibial implant insimulating the articulating motion of an anatomical knee joint.

These femoral and tibial implants, in combination with ligaments andmuscles, attempt to duplicate natural knee motion as well as absorb andcontrol forces generated during the range of flexion. In some instanceshowever, it may be necessary to replace or modify an existing femoraland/or tibial implant. Such replacements are generally referred to asrevision implants.

To prepare a femur and tibia for such a knee replacement and form anengagement with femoral and tibial implants, the femur and tibia bonesmust be cut in very specific and precise ways and at very specific andprecise angles and locations, so that the prepared bone will properlyengage with and be secured to the corresponding implants. In order tomake these cuts properly, a surgeon traditionally uses a jig, orsurgical cutting guide as known to those skilled in the field, which canbe removably attached or secured to the bone, such that slots, orguides, in the jig facilitate the precise cuts necessary to secure thecorresponding implants.

The phrase “jig” as used herein, shall thus refer broadly to a surgicalcutting guide, that may be configured and arranged to be fixed orattached to a bone, or may be secured adjacent to a bone or other tissueto be cut by a surgeon and identify a relative location, angle and/orcutting plane that a surgeon should cut on the adjacent bone or tissue,as known in the art. A jig may include predetermined slots and/orcutting surfaces to identify where a surgeon should cut the adjacentbone or tissue, wherein such cuts may correspond to a shape of asurgical implant that may be attached to the cut bone or tissue.

Therefore, there is a need for a system that can identify a mechanicalaxis of the femur, or the center of the femoral head, without having toinsert a rod into the femur or expose the hip socket, which can lead tocomplication during surgery and require significant surgery time andadditional recovery time for the patient.

Accordingly, there is a need for a system and method of utilizing avirtual or holographic mechanical axis or surgical instrument that couldfacilitate increased accuracy and precision of required or desired bonecuts.

The phrases “virtual jig,” “holographic jig,” “virtual axis,” or“holographic axis” as used herein, shall thus refer broadly to anyvisual rendering or projection representing an actual physical jig, ormechanical or anatomical axis of a bone, as the case may be, havingsome, all, or mostly all, of the same visual characteristics of thephysical jig or axis, as the case may be, including the visualappearance of the same size and shape as the physical objects beingvirtually represented, as known in the art.

The features and advantages of the present disclosure will be set forthin the description which follows, and in part will be apparent from thedescription, or may be learned by the practice of the present disclosurewithout undue experimentation. The features and advantages of thepresent disclosure may be realized and obtained by means of theinstruments and combinations particularly pointed out in the appendedclaims. Any discussion of documents, acts, materials, devices, articlesor the like which has been included in the present specification is notto be taken as an admission that any or all of these matters form partof the prior art base, or were common general knowledge in the fieldrelevant to the present disclosure as it existed before the prioritydate of each claim of this application.

BRIEF DESCRIPTION OF THE DRAWINGS

The features and advantages of the disclosure will become apparent froma consideration of the subsequent detailed description presented inconnection with the accompanying drawings in which:

FIG. 1a is a schematic rendering of a mixed reality system of thepresent disclosure;

FIG. 1b is a schematic rendering of a view through a mixed realitydisplay of an embodiment of the present disclosure;

FIG. 1c is a schematic rendering of a view through a mixed realitydisplay of another embodiment of the present disclosure;

FIG. 1d is a schematic rendering of a view through a mixed realitydisplay of a further embodiment of the present disclosure;

FIG. 2 is a perspective view of a 3D spatial mapping camera of anotherembodiment of the present disclosure;

FIG. 3 is a perspective view of a 3D spatial mapping camera of anotherembodiment of the present disclosure;

FIG. 4 is a perspective view of a 3D spatial mapping camera of anotherembodiment of the present disclosure;

FIG. 5 is a perspective view of a 3D spatial mapping camera of anotherembodiment of the present disclosure;

FIG. 6 is a perspective view of a 3D spatial mapping camera of anotherembodiment of the present disclosure;

FIG. 7 is a perspective view of a 3D spatial mapping camera of anotherembodiment of the present disclosure;

FIG. 8 is a perspective view of a 3D spatial mapping camera of anotherembodiment of the present disclosure;

FIG. 9 is a perspective view of a 3D spatial mapping camera of anotherembodiment of the present disclosure;

FIG. 10 is a perspective view of another embodiment system of thepresent disclosure;

FIG. 11 is a perspective view of an optical label of another embodimentof the present disclosure; and

FIG. 12 is a perspective view of another embodiment of the presentdisclosure.

DETAILED DESCRIPTION

For the purposes of promoting an understanding of the principles inaccordance with the disclosure, reference will now be made to theembodiments illustrated in the drawings and specific language will beused to describe the same. It will nevertheless be understood that nolimitation of the scope of the disclosure is thereby intended. Anyalterations and further modifications of the inventive featuresillustrated herein, and any additional applications of the principles ofthe disclosure as illustrated herein, which would normally occur to oneskilled in the relevant art and having possession of this disclosure,are to be considered within the scope of the disclosure claimed.

It must be noted that, as used in this specification and the appendedclaims, the singular forms “a,” “an,” and “the” include plural referentsunless the context clearly dictates otherwise.

In describing and claiming the present disclosure, the followingterminology will be used in accordance with the definitions set outbelow.

As used herein, the terms “comprising,” “including,” “containing,”“characterized by,” and grammatical equivalents thereof are inclusive oropen-ended terms that do not exclude additional, unrecited elements ormethod steps.

As used herein, the terms “virtual” and “hologram” are usedinterchangeably, and grammatical equivalents thereof are inclusive oropen-ended terms that do not exclude additional, unrecited elements ormethod steps. These terms are used to describe visual representations ofan actual physical object, device or element, having some, all, ormostly all, of the same visual characteristics of the physical device,including the visual appearance of the same size and shape as thephysical object device or element being virtually represented.

Applicant has discovered a novel system and method for generating andusing a virtual axis and/or virtual instrument, in a surgical procedure,for example, in a knee or tibial implant procedure, or other desiredsurgical procedure.

The phrase “virtual system” as used herein, shall refer broadly to anysystem capable of generating or creating a simulated or virtualrendering or projection of physical or structural objects, devices orfeatures virtually identical or, substantially identical to an actualphysical device, object, instrument or other structure, as known in theart. A virtual system may also include a device, mechanism, orinstrument capable of projecting or displaying the desired simulated orvirtual rendering or projection of physical or structural featuresvirtually identical or substantially identical to an actual physicaldevice or features or portions thereof. A virtual system may also enablea user to manipulate, move and/or modify the simulated or virtualrendering or projection.

The phrase “mixed or augmented reality system” as used herein, shallrefer broadly to any system capable of generating or creating asimulated or virtual rendering or projection of physical or structuralobjects, devices or features virtually identical or substantiallyidentical to an actual physical device, instrument or other structure orfeature or portions thereof, as known in the art. A mixed or augmentedreality system may also include a device, mechanism, or instrumentcapable of projecting or displaying the desired a simulated or virtualrendering or projection of physical or structural features virtuallyidentical or substantially identical to an actual physical deviceoverlaid or concurrently with actual physical structures, objects,mechanisms or devices in reality, thus incorporating the virtualrendering or projection in real world settings with actual physicalelements. A mixed or augmented reality system may also enable a user tomanipulate, move and/or modify the simulated or virtual rendering orprojection.

The phrase “mixed or augmented reality instrument” as used herein, shallrefer broadly to any device, mechanism or instrument used in a mixed oraugmented reality system, including a device capable of generating orcreating a simulated or virtual rendering or projection of physical orstructural elements and/or features virtually identical or substantiallyidentical to an actual physical device, object, instrument or otherphysical structure, as known in the art. A mixed or augmented realityinstrument may also be capable of projecting or displaying the desiredsimulated or virtual rendering or projection of physical or structuralfeatures or elements virtually identical or substantially identical toan actual physical device or object overlaid or concurrently with actualphysical structures, object, mechanism or devices in reality, thusincorporating the virtual rendering or projection in real world settingswith actual physical elements or features. A mixed or augmented realityinstrument may also enable a user to manipulate, move and/or modify thesimulated or virtual rendering or projection.

The phrase “holographic representation” as used herein, shall referbroadly to a visual rendering or projection representing an actualphysical device, object or element or portion thereof, having some, all,or mostly all, of the same visual characteristics of the correspondingphysical device, object or element, including the visual appearance ofthe same size and shape as the physical objects being virtuallyrepresented, as known in the art.

Referring to the drawings, where like numbers represent like elements,FIG. 1a , in a disclosed embodiment, illustrates a mixed or augmentedreality system, generally indicated at 100, which can be used toproduce, or display, a desired mixed or augmented reality instrument,such as an anatomical or mechanical axis of a bone in a display to asurgeon or user, or stated another way, that is visible andmanipulatable by a surgeon or user. The mixed or augmented realitysystem 100 may also enable a user to activate or deactivate, in full orin part, the marker, such as an axis, or virtual instrument(s), making avirtual axis appear or disappear, as desired in a mixed reality assistedsurgery, for example.

The mixed or augmented reality system 100 may include a mixed oraugmented reality headset 102 which may include a transparent or mostlytransparent viewer 104, or display, which can be suspended or positionedin front of a user's eyes. In alternative embodiments, the viewer 104,or mixed reality display, may be a stand-alone device detached andseparate from the surgeon, or may be mounted in another desired locationin an operating room or other desired location. The headset 102 mayinclude a headband 106 attached to the viewer 104, which may be used tosecure the headset 102 to a user's head 108, thereby securing the viewer104 in place in front of the user's eyes.

The transparent viewer 104 may be configured to project, or otherwisemake viewable, on an interior surface of the viewer 104, a holographicvirtual image or images, such as a virtual instrument, for example, ananatomical or mechanical axis of a bone, which may be positionallymanipulated by the user, surgeon, third party or remote system, such asa remote computer system. For the purpose of this disclosure the term“mechanical axis” as used herein shall be defined broadly in referenceto a bone having a proximal joint and a distal joint, as a straight lineconnecting the joint center points of the proximal and distal joints inthe frontal or sagittal planes. The headset 102 may be configured toview holographic images or, alternatively, the holographic images may beturned off and the user wearing the headset 102 may be able to view thesurrounding environment through the transparent viewer 104,unobstructed. As such, a user, such as a surgeon for example, can wearthe mixed or augmented reality headset 102 and then can choose toactivate a holographic image to aide in facilitating a surgicalprocedure and then shut off the holographic image in order to performthe surgical procedure un-obscured, visually.

One embodiment of the disclosed headset 102 may be a product created andmanufactured by Microsoft, known as the HoloLens® mixed or augmentedreality system, or any suitable mixed or augmented reality system forgenerating virtual images viewable by a user or surgeon. Headset 102 maybe a conventional “off the shelf” product with a built-in platform thatenables all of the features described herein with respect to the headset102.

In addition to identifying axes of a bone, in alternative embodiments,the headset 102, such as a Microsoft HoloLens product, can be loaded orpreloaded with all desired or required virtual instruments, includingvirtual jigs or surgical cutting guides, virtual drill bits, and/or avirtual target which can identify relative locations of a plurality ofholes to be drilled by a surgeon to facilitate the fastening of a jig orother device onto a desired bone at the proper desired location, and anyother desired virtual instruments or holograms. The Microsoft HoloLensproduct and its capabilities and features, or any suitable mixed oraugmented reality system such as is described herein with respect to theheadset 102, are known to those skilled in the art.

The mixed reality system 100 may also include a computer or computersystem 200 having enabling software to communicate with the headset 102,by both receiving information from the headset 102 and transmitting dataand virtual images to the headset 102. It is therefore to be understood,by way of the circuit diagram and dashed lines shown in FIG. 1, thatheadset 102 is electronically connected to the computer system 200 and a3D spatial mapping device or camera 300. The 3D spatial mapping camera300 is electronically connected to the headset 102 and the computersystem 200, as represented by the dashed lines shown in FIG. 1. Whilethe 3D spatial mapping camera 300 is electronically connected to theheadset 102, the 3D spatial mapping camera 300 may be separate from andnot mechanically connected to the headset 102.

The mixed reality system 100 may also include a 3D spatial mappingcamera 300. One embodiment of the disclosed spatial mapping camera 300may be a product created and manufactured by Microsoft, known as theAzure Kinect®, or any suitable 3D spatial mapping camera or LiDARScanner capable of continuous 3D mapping and transition corresponding 3Dimages, such as bones, anatomy, or other desired 3D objects. The spatialmapping camera 300 may be a conventional “off the shelf” product with abuilt-in platform that enables all of the features described herein withrespect to the spatial mapping camera 200. Furthermore, the spatialmapping camera 200, such as a Microsoft Azure Kinect product, can beloaded or preloaded with all necessary software to enable wirelesscommunication between the spatial mapping camera 300 and the computersystem 200 and/or the headset 102. The Microsoft Azure Kinect productand its capabilities and features, or any suitable 3D spatial mappingcamera such as is described herein with respect to the spatial mappingcamera 300, are known to those skilled in the art.

The headset 102, computer system 200 and spatial mapping camera 300, maybe programmed and configured to enable a surgeon 107 to see andmanipulate a virtual, or holographic target or axis, with respect apatient's bone 400, anatomical, or any other desired location, which mayreceive a surgical implant. The headset 102, computer system 200 andspatial mapping camera 300 may communicate with one another via a localnetwork connection, W-Fi, Bluetooth® wireless technology, or any otherknown wireless communication signal.

Specifically, the spatial mapping camera 300, that may be programed tocommunicate with the computer system 200 having enabling software, mayutilize such enabling software to map the bone 400 and generate map datarepresentative of a three dimensional map of a surface of the bone 400,or other desired anatomy, to help identify the location and orientationof an anatomical or mechanical axis of a bone, which can then facilitatethe proper placement of a jig, implant or other device, to the bone 400,prior to cutting the knee.

The mixed reality system 100 may also include a marker 500. The marker500 may be attached or otherwise fixed to a distal end of the bone 400,which may be exposed. While a plurality of markers 500 may be used, ifdesired, a single, exclusive marker 500, may also be used in thisdisclosed embodiment. Bone 400 is a femur in the illustrated embodimentof FIG. 1, however, the marker 500 may be fixed to any desired bonehaving in inaccessible pivot point or fulcrum. The “distal end sectionor portion” may include the terminal distal end, the distal end sectionor portion as used herein being defined broadly as a distal half of thebone 400. The marker 500 may include a scannable, visual, optical label502, such as a QR code.

The surgeon may attach the marker 500 to an exposed distal end sectionof the bone 400, at a predetermined or desired location. The spatialmapping camera 300 may spatially map the marker 500 and the exposed bone400 to map the surface of the exposed bone and relative location of themarker 500, including the optical label 502.

The optical label 502 may include a QR code or other opticalidentifiers, such as shapes having distinct corners, which can beidentified by the 3D spatial mapping camera 300 and the location of theoptical label identified and stored.

The surgeon may then rotate the bone 400, with the marker 500 fixedthere to, in a non-liner path 504, such as a circular path, ovular pathor another desired, predetermined, or random non-liner path. The bone400 may be rotated about a fulcrum or pivot point that is not exposed tothe surgeon's view, for example the socket joint of the hip, but mayalso include any desired pivot point or fulcrum that may not be exposedto the surgeon, such as a shoulder joint, for example.

At specific time intervals, the 3D spatial mapping camera 300 maycapture an image of the location of the marker 500 and optical label 502and store the location data. The time intervals between the plurality ofcaptures of the location data may be predetermined and consistent timeintervals, at random time intervals, or at time intervals controlled bythe surgeon. Additionally, the number of image captures by the 3Dspatial mapping camera may be set at a predetermined number, or at arandom number, or at a number that is determined by the surgeon. Forexample, the 3D spatial mapping camera may capture an image every secondfor 30 seconds, or may capture 10 images every second for 3 seconds, orany other desired number of images over any desired time intervals.

After the plurality of image captures of the marker 500 and opticallabel 502, the images and location data may be sent to the computersystem 20 which may then process the image and location data, and usinga preloaded algorithm, may generate the mechanical axis 402 of the bone400.

Additionally, the computer system 200, may also process the image andlocation data, and using a preloaded algorithm, may generate amechanical center 404 of the femoral head of the bone 400.

The computer system 200, may also process the image and location data,and using a preloaded algorithm, may generate a cut line 406, or cutplane, that can be used by a surgeon to identify where the bone 400should be cut in preparation of the attachment of an implant or otherdesired surgical device. The cut plane 406 forms an angle with themechanical axis 402 of the bone 400 and may, for example, be orientedorthogonal to the mechanical axis of the bone.

The computer system 200 may then send data related to the mechanicalaxis 402, mechanical center 404, cutting line or plane 406, and/orvirtual jig 410, to the headset 102, which can then utilize the dataprovided by the computer to provide a virtual image of the mechanicalaxis 402, mechanical center 404, the cutting line or plane 406, and/orvirtual jig 410, relative to the bone 400, which can then be displayedon the viewer 104, that visually appears to overlay bone 400 at aposition and in an orientation of the mechanical axis 402 when viewedwith the mixed reality display. The surgeon can then view the virtualmechanical axis 402, mechanical center 404, the cutting line or plane406, and/or virtual jig 410, in their accurate location with respect tothe physical bone 400, which can be seen simultaneously with the virtualimages. The surgeon can then cut the bone 400, attach a device orimplant to the bone 400 or otherwise manipulate the bone using theaccurately displayed virtual mechanical axis 402, mechanical center 404,and/or the cutting line or plane 406.

FIG. 1b illustrates a schematic view of a perspective through the viewer104 of the mixed reality headset 102, representing the view of a wearerof the mixed reality headset 100 during a procedure. As shown, the viewcan display a virtual mechanical axis 402 overlaid on the actualphysical bone 400, enabling the wearer to see or view the virtualmechanical axis 402 simultaneously with the bone 400, marker 500 andlabel 502.

FIG. 1c illustrates a schematic view of a perspective through the viewer104 of the mixed reality headset 102, representing the view of a wearerof the mixed reality headset 100 during a procedure. As shown, the viewcan display a virtual mechanical axis 402 overlaid on the actualphysical bone 400, enabling the wearer to see or view the virtualcutting line or plane 406 simultaneously with the bone 400, marker 500and label 502.

FIG. 1d illustrates a schematic view of a perspective through the viewer104 of the mixed reality headset 102, representing the view of a wearerof the mixed reality headset 100 during a procedure. As shown, the viewcan display a virtual mechanical axis 402 overlaid on the actualphysical bone 400, enabling the wearer to see or view a virtual jig 410simultaneously with the bone 400, marker 500 and label 502.

In another embodiment, as shown in FIG. 2, the 3D spatial mapping cameramay be fixed or removably attached to a stand 600. The stand 600 may beanchored or fixed to a desired location in the operating room, or thestand may be mobile, utilizing a plurality of wheels 602, for example.

In another embodiment, as shown in FIG. 3, the 3D spatial mapping camera300 may be fixed or removably attached to a stand 700. The stand 700 maybe anchored or fixed to a desired location in the operating room, or thestand may be mobile, utilizing a plurality of wheels 702, for example.The stand 700 may also include a flexible arm 704 that may allow a userto manipulate the positioning of the 3D spatial mapping camera 300.

In another embodiment, as shown in FIG. 4, the 3D spatial mapping camera300 may be sized and configured to be fixed or removably attached to astrap 800. The strap 800 may be adjustable and can be configured to beworn by a user around their head, chest or other desired part of thebody.

In another embodiment, as shown in FIG. 5, the 3D spatial mapping camera300 may be sized and configured to be fixed or removably attached to aheadset 900. The headset 900 may be worn on the head of a user as partof eye protection or glasses or as part of a mixed reality headset.

In another embodiment, as shown in FIG. 6, the 3D spatial mapping camera300 may be sized and configured to be attached or integrated into alight source panel 1000 used during an operation. The illustration ofFIG. 6 shows different locations on the light source panel 1000 thatcould incorporate the 3D spatial mapping camera 300 (various locationidentified by the arrows). For example, the 3D spatial mapping cameramay be incorporated into a handle 1002, a light source 1004, a frame1006, or a sleeve 108 which may provide a sterile barrier for the handle1002.

In another embodiment, as shown in FIG. 7, the 3D spatial mapping camera300 may be fixed or removably attached to an operating table 1100 via amount 1102. The mount 1102 may be fixed or attached to the operatingtable at any desired location and may be adjustable in height and/ordirection, enabling a surgeon or user to manipulate the angle of viewand location of the 3D spatial mapping camera 300.

In another embodiment, as shown in FIG. 8, the 3D spatial mapping camera300 may be worn by a surgeon 1200 or an assistant 1202 using any of theaforementioned 3D spatial mapping camera embodiments. The 3D spatialmapping camera may be worn by a user inside or outside of a sterilefield and could be worn in a way that would conform to specificprocedural requirements a particular surgery.

In another embodiment, as shown in FIG. 9, the 3D spatial mapping camera300 may be fixed or removably attached to a variety of differentlocations within an operating room 1300. For example, the 3D spatialmapping camera 300 may be attached or fixed to a wall 1302, a floor1304, a wall panel 1306, a wall fixture 1308, a ceiling 1310 or anyother desired location. Additionally, multiple 3D spatial mappingcameras 300 can be used simultaneously or in concert with one anotherwhere each 3D spatial mapping camera 300 may be mounted in differentlocations in the operating room 1300 or at the same location.

In another embodiment, as shown in FIG. 10, the system 100 may includeall of the same components as discuss and disclosed in FIG. 1, but mayalso include a plurality of markers 500 attached to the exposed portionof the bone 400. As disclosed above with respect to a single marker 500,the use of multiple markers 500 may provide additional data that may becaptured by the 3D spatial mapping camera 300 and processed by thecomputer system 200 to locate the mechanical axis 402, mechanical center404, and/or the cut line or plane 406.

In another embodiment, as shown in FIG. 11, the earlier disclosedoptical labels 502 may be QR codes, as shown in examples 1400, 1402, and1404. QR codes 1400, 1402, and 1404 may include various shapes and be ofvarious desired sizes, and may include a plurality of corners, or sharppoints that may be scanned their images captured by the 3D spatialmapping camera 300 to provide additional location data that may beprocessed by the computer system 200 as discussed in the other disclosedembodiments. Additionally, the markers 500 and/or optical labels 502 maybe may of reflective material, of any desired color, may emit light, maybe formed of any desired shape or of any desired size, to facilitate thecapturing of an image to provide location data.

FIG. 12 illustrates that, additional uses of the disclosed system 100and accompanying embodiments, may include a bone 1500 having a fracture1501. By using the above-mentioned system and method a surgeon mayidentify the location of the fracture 1501, being a fulcrum or pivotpoint as described above, or other discontinuity. The system 100 canthen be used, using the same location determining methodology and systemdescribed above, to locate the location of a stabilizing rod 1502 andthe corresponding location of where screws 1504 would need to be locatedto be properly and accurately received by the rod 1502. This can beparticularly useful in circumstances where critical locations are notvisibly exposed to the surgeon.

It is to be understood that the various embodiments disclosed anddescribed above and shown in the accompanying figures, may beinterchangeably used together, independently or in any desiredcombination of disclosed features.

In the foregoing Detailed Description, various features of the presentdisclosure are grouped together in a single embodiment for the purposeof streamlining the disclosure. This method of disclosure is not to beinterpreted as reflecting an intention that the claimed disclosurerequires more features than are expressly recited in each claim. Rather,as the following claims reflect, inventive aspects lie in less than allfeatures of a single foregoing disclosed embodiment. Thus, the followingclaims are hereby incorporated into this Detailed Description of theDisclosure by this reference, with each claim standing on its own as aseparate embodiment of the present disclosure.

It is to be understood that the above-described arrangements are onlyillustrative of the application of the principles of the presentdisclosure. Numerous modifications and alternative arrangements,including but not limited to combinations of elements and/or featuresfrom the various disclosed embodiments may be devised by those skilledin the art without departing from the spirit and scope of the presentdisclosure and the invention is intended to cover such modifications,arrangements and combinations. Thus, while the present disclosure hasbeen shown in the drawings and described above with particularity anddetail, it will be apparent to those of ordinary skill in the art thatnumerous modifications, including, but not limited to, variations insize, materials, shape, form, function and manner of operation, assemblyand use may be made without departing from the principles and conceptsset forth herein.

1. A system for determining a location for a surgical procedure,utilizing: a 3D spatial mapping device, wherein the 3D spatial mappingdevice is configured to generate map data representative of a threedimensional map a surface of a bone, a marker attached to a distal endportion of the bone; wherein the 3D spatial mapping device is configuredto capture a plurality of images of the marker as the bone is rotated ina non-linear path, wherein the images include marker data identifying amarker location of the marker relative to the three dimensional map dataof the surface of the bone; a computer system that receives the map dataand marker data from the images captured by the 3D spatial mappingdevice and determines a position and orientation of a mechanical axis ofthe bone relative to the bone; a mixed reality display, wherein thecomputer system is configured to send the position and orientation ofthe mechanical axis to the mixed reality display and the mixed realitydisplay is configured to provide a virtual image of the mechanical axisof the bone that visually appears to overlay the bone at the positionand in the orientation of the mechanical axis when viewed with the mixedreality display.
 2. The system of claim 1, wherein the mixed realitydisplay is a mixed reality headset.
 3. The system of claim 1, whereinthe distal end section of the bone is visually exposed.
 4. The system ofclaim 1, wherein the marker includes a QR code.
 5. The system of claim1, wherein the system includes only a single marker.
 6. The system ofclaim 1, wherein the computer system that receives the data from theimages captured by the 3D spatial mapping device and determines alocation of a mechanical center of a head of the bone.
 7. The system ofclaim 1, wherein the computer system that receives the data from theimages captured by the 3D spatial mapping device and determines alocation of a cut line or cut plane.
 8. The system of claim 1, whereinthe 3D spatial mapping device is configured to capture a plurality ofimages of the marker as the bone is rotated in a non-linear path about apivot point, or fulcrum, that is not visual exposed.
 9. The system ofclaim 1, wherein the system may include a plurality of markers attachedto the bone.
 10. The system of claim 1, wherein the 3D spatial mappingdevice is configured to capture the plurality of images at predeterminedtime intervals during rotation of the bone.
 11. A system determining alocation for a surgical procedure, comprising: a 3D spatial mappingdevice, wherein the 3D spatial mapping device is configured to map asurface of a bone, a marker, comprised of a QR code, attached to adistal end portion of the bone; wherein the 3D spatial mapping device isconfigured to capture a plurality of images of the marker relative tothe bone as the bone moved relative to the 3D spatial mapping device,wherein the images include marker data identifying a location of themarker relative to the bone; a computer system that receives the markerdata from the images captured by the 3D spatial mapping device anddetermines a position and orientation of a mechanical axis of the bonerelative to the bone; a mixed reality display, wherein the computersystem is configured to send the position and orientation of themechanical axis relative to the bone to the mixed reality display andthe mixed reality display is configured to provide a virtual image ofthe mechanical axis of the bone that visually appears to overlay thebone at the position and in the orientation of the mechanical axis whenviewed with the mixed reality display.
 12. The system of claim 11,wherein the mixed reality display is a mixed reality headset.
 13. Thesystem of claim 11, wherein the distal end section of the bone isvisually exposed.
 14. The system of claim 11, wherein the movement ofthe bone occurs along a non-linear path.
 15. The system of claim 11,wherein the system includes only a single marker.
 16. The system ofclaim 11, wherein the computer system that receives the data from theimages captured by the 3D spatial mapping device and determines alocation of a mechanical center of a head of the bone.
 17. The system ofclaim 11, wherein the computer system that receives the data from theimages captured by the 3D spatial mapping device and determines alocation of a cut line or cut plane.
 18. The system of claim 11, whereinthe 3D spatial mapping device is configured to capture a plurality ofimages of the marker as the bone is rotated in a non-linear path about apivot point, or fulcrum, that is not visual exposed.
 19. The system ofclaim 11, wherein the system may include a plurality of markers attachedto the bone.
 20. The system of claim 11, wherein the 3D spatial mappingdevice is configured to capture the plurality of images at predeterminedtime intervals during rotation of the bone.
 21. A system for determininga location for a surgical procedure, utilizing: a 3D spatial mappingcamera, wherein the 3D spatial mapping camera is configured to map abone, a marker attached to a distal end section of the bone; wherein the3D spatial mapping camera is configured to capture a plurality of imagesof the marker as the bone is moved, wherein the images include dataidentifying a location of the marker; a computer system that receivesthe data from the images captured by the 3D spatial mapping camera anddetermines a location of a cut plane on the bone; and a mixed realitydisplay, wherein the computer system is configured to send the locationof the mechanical axis to the mixed reality display and the mixedreality display is configured to provide a virtual image of the cutplane that visually appears to overlay the bone at a position and in anorientation relative to the bone when viewed with the mixed realitydisplay.
 22. The system of claim 21, wherein the mixed reality displayis a mixed reality headset.
 23. The system of claim 21, wherein thedistal end section of the bone is visually exposed.
 24. The system ofclaim 21, wherein the marker includes a QR code.
 25. The system of claim21, wherein the system includes only a single marker.
 26. The system ofclaim 21, wherein the computer system that receives the data from theimages captured by the 3D spatial mapping camera and determines alocation of a mechanical center of a head of the bone.
 27. The system ofclaim 21, wherein the computer system that receives the data from theimages captured by the 3D spatial mapping camera and determines alocation of a mechanical axis of the bone.
 28. The system of claim 21,wherein the 3D spatial mapping camera is configured to capture aplurality of images of the marker as the bone is rotated in a non-linearpath about a pivot point, or fulcrum, that is not visual exposed. 29.The system of claim 21, wherein the system may include a plurality ofmarkers attached to the bone.
 30. (canceled)
 31. (canceled) 32.(canceled)
 33. (canceled)
 34. (canceled)
 35. (canceled)
 36. (canceled)37. (canceled)
 38. (canceled)
 39. (canceled)
 40. (canceled) 41.(canceled)
 42. A system for determining a location for a surgicalprocedure, utilizing: a 3D spatial mapping camera, wherein the 3Dspatial mapping camera is configured to map a bone, a single markerhaving a QR code attached to a distal end section of the bone, whereinthe distal end section of the bone is visually exposed; wherein the 3Dspatial mapping camera is configured to capture a plurality of images ofthe marker as the bone is rotated in a non-linear path about a pivotpoint, or fulcrum, that is not visual exposed, wherein the imagesinclude data identifying a location of the marker, and wherein the 3Dspatial mapping camera is configured to capture the plurality of imagesat predetermined time intervals during rotation of the bone; a computersystem that receives the data from the images captured by the 3D spatialmapping camera and determines a location of a cut plane on the bone; anda mixed reality display, the mixed reality display being a mixed realityheadset, wherein the computer system is configured to send the locationof the mechanical axis to the mixed reality display and the mixedreality display is configured to provide a virtual display of themechanical axis of the bone, and wherein the computer system alsodetermines a location of a mechanical center of a head of the bone.